Remove use of distinct in event listing #649
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
voneiden
force-pushed
the
link-1408-optimize-distinct
branch
2 times, most recently
from
1c27f6a
to
19b0627
Compare
Codecov Report
@@ Coverage Diff @@
## develop #649 +/- ##
===========================================
+ Coverage 75.45% 76.06% +0.60%
===========================================
Files 237 239 +2
Lines 17297 17357 +60
===========================================
+ Hits 13052 13203 +151
+ Misses 4245 4154 -91
... and 4 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
voneiden
force-pushed
the
link-1408-optimize-distinct
branch
from
19b0627
to
56bee3e
Compare
|
Yritetty tarkistaa seuraavat relaatiot Events many-to-manyin_language Events one-to-manyvideos |
voneiden
force-pushed
the
link-1408-optimize-distinct
branch
from
56bee3e
to
a00fc95
Compare
danipran
reviewed
Aug 4, 2023
danipran
approved these changes
Aug 4, 2023
voneiden
force-pushed
the
link-1408-optimize-distinct
branch
3 times, most recently
from
d867d2c
to
e47cf32
Compare
voneiden
commented
Aug 7, 2023
voneiden
force-pushed
the
link-1408-optimize-distinct
branch
from
e47cf32
to
8f60a93
Compare
charn
reviewed
Aug 8, 2023
charn
approved these changes
Aug 8, 2023
The use of distinct is extremely slow. It's use can be avoided in one-to-many and many-to-many relations by using exclude(~Q(relation=something). When excluding relations, Django automatically uses SQL EXISTS. Negating the Q statement will essentially create a NOT NOT EXISTS which the database then optimizes into plain EXISTS. This appears to be still notably faster than using filter(Exists(subquery)) which creates an INNER JOIN inside the exists clause. Ref LINK-1408
voneiden
force-pushed
the
link-1408-optimize-distinct
branch
from
48c5714
to
3324685
Compare
|
Kudos, SonarCloud Quality Gate passed! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The use of distinct is extremely slow in the event listing.
The need for distinct can be replaced in one-to-many and many-to-many relations by using
exclude(~Q(relation=something). When excluding relations, Django automatically uses SQL EXISTS. Negating the Q statement will essentially create a NOT NOT EXISTS which the database then optimizes into plain EXISTS.DISTINCT can be avoided by utilizing EXISTS queries in filters that can create duplicate rows. Additionally some filter queries on m2m relations can be made still significantly faster if we target the through table directly to avoid unnecessary INNER JOIN by django.